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ABSTRACT 

Unanticipated obstacles and difficulties, endeeic to 
industrial research in coeplex sociocultural settings, indicate that 
reliable conclusions concerning industrial life are probleeatic. 
Discussion of saspling biases reveals that error of unknovn 
proportions is introduced into the research process: first, vhen 
decisions about whom to study are sade; second, vhen "entree" Is 
attempted ; and third, vhen individual response cooperation is 
solicited. The authors conclude that only vhen data are cumulated 
across a sizable number of independent studies can reliable 
conclusions ibout industrial behavior be sade. Also included are a 
six^ites bibliography and a distribution list. (Author/BP) 
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Unanticipated obstacles and difficulties, endemic to industrial research 
in complex soclo-cultural settings, indicate that .reliable conclusions 
concerning industrial life are problematic. Discussion of sampling biases 
reveals that error of unknown proportions is introduced into the research 
process: first » when decisions about whom to study are made; second » when 
entree is attempted; and third, when individual response cooperation is 
solicited. The authors conclude that only when data are cumulated across a 
sizable number of independent studies can reliable conclusions about industrial 
behavior be made. 
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DON'T SHAKE THE CHAUFFEUR^S HAND: 
TOWARD RELIABLE KNOWLEDGE ABOUT INDUSTRIAL LIFE 
R. Alan Hedley 
University of Victoria 
and Thomas C. Taveggia 
University of California, Irvine 

In January, 1969, two young North American sociologists stepped off a 
train in Birmingham, England, on their way to establish research entry into 
a large British manufacturing firm. They were met, as is the custom, by a 
uniformed chauffeur who was to take them to the company* s head office. 
After first establishing proper identity, the chauffeur reached out his 
gloved hand in order to take the briefcase held by one of the researchers. 
Engaged in conversation and, for the moment,- misinterpreting this gesture , 
the sociologist grabbed the chauffeur's hand > and shook it. Considerable 
confusion followed this brief encounter and,' in the end, the chauffeur 
refused to converse with either researcher. 

Although not in any catastrophic sense, this minor incident did affect 
the research in which the two sociologists were engaged. In firms they Had 
researched in Britain prior to this one, they had learned that the chauffeur 
could be an invaluable guide to the culture of the firm. Usually, he knew 
of the concerns of senior management an^ how they were likely to view the 
anticipated research, as well as how the research would be received by other 
subgroups within the plant. These important bits of information were lost to 
the researchers when one inadvertantly violated the chauffeur's expectations 
by shaking his gloved hand. 

Encounters like this one are typically not discussed in industrial 
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research reports (dissertations excepted). But, the experiences of most 
industrial researchers are probably filled with similar examples of unanticipated 
factors arising to disrupt the planned sequence of their research. In fact, 
we are convinced that snags, difficulties, and obstacles are endemic to 
research in complex industrial settings. Furthermore, and in a more polemical 
vein, we suggest that the findings of single research studies consequently ^ 
are uninterpretable; that reliable conclusions about industrial life will 
emerge only as the findings of numerous and independent researches are pooled* 
Evaluating Industrial Research 

It is a truism that normative standards govern research in all scientific 
fields. Criteria not only guide the design and conduct of research, but also 
its interpretation and evaluation. This point is well illustrated in the field 
of industrial research where a number of standard criteria are employed as 
"bench marks** to assess research. Included here would be criteria like theo- 
retical significance, "validity*' and "reliability" of measures, sample repre- 
sentativeness, and the appropriateness of analytic techniques for the data at 
nand • 

Although a. thorough evaluation of any research requires evidence on these 
and other criteria, it is a reasonable premise that "...if the sampling 
stability and bias of a finding are unknown or are known to be adverse, little 
else can be supported unambiguously* That is, estimation of random error 
and sampling bias is a necessary (but not sufficient) condition for adequate 
evaluation" (Finifter, 1972:115). 

Unfortunately, sampling bias is more often than not unknown in industrial 
research. Irrespective of the attention devoted to sampling considerations 
during the design phase of industrial research, experience reveals that 
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Industrial researchers typically encounter difficulties and problems In the 
course of securing their samples chat Introduce unknovm, and hence unpre-* 
dlctable, sources of bias Into their research. 
Sampling Bias In Industrial Research 

Prior to entering the field. Industrial researchers like other social 
scientists face the decision of whom to study. Ideally, this means setting 
up a representative random sampling plan. However, In most Industrial studies 
this Ideal Is compromised by concern over which firms, given time and money 
considerations, are most likely to be supportive of the anticipated research 
(Delany, 1960:449-51)9 ^^d by difficulties actually encountered in securing 
both entree to selected firms and the cooperation of industrial workers (see, 
for example, Scott, 1969). The magnitude of these problems is clearly indicated 
by the fact that very few industrial studies exist that are based on random 
samples. 

Our experiences in conducting an attitu^nal survey of approximately 
5300 British industrial workers are illustrative of the kinds of snags, 
difficulties, and obstacles that compromise the "best laid plans...** For 
example, at the outset of our research we had hoped to design our study in 
such a manner as to ensure that our findings would be representative of the 
experiences of all British workers. Initially, we envisioned a two-stage 
random sampling plan which was to Involve sampling industrial organizations 
and then individuals within selected organizations. We realized quickly, 
ho«/ever, that such a plan was impractical given our limited time and resources, 
and the reputedly high refusal rate of industrial organizations in Britain. 
Consequently, we were compelled to make an alternative sampling decision 
which Involved establishing a list of industrial firms known to be supportive 
of social research and also who employed diverse groups of industrial workers. 
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Our plan simply was to select organizations purposlvely from this list to 
provide us with a large and heterogeneous sample in terms o£ selected study 
variables. Thus, we selected a total of twelve firms. 

Immediately, it can he, seen that we were compelled to confine our attention 
to a group of firms which could very probably jeopardize the generalizability 
of our findings » for it is likely that organization which have sponsored 
previous research are different in significant respects from those which are 
generally non-supportive. At any rate, it is very probable that individuals 
working in these organizations differ substantially from individuals in firms 
which have not sponsored previous research. Thus, even before entering the 
field, unknown sampling biases were introduced into our research. 

Of course, our experiences in this respect are not unique. Similar 
problems have certainly been noted in the case of the Hawthorne studies where, 
over a period of at least eight years, investigators, first from the National 
Research Council and then from the Harvard Graduate School of Business Admin<- 
istration, measured the attitudes and behavior of industrial workers. * Similar 
instances of a possible ^^Hawthorne effect** are noted throughout the literature. 
For example, the Harwood Manufacturing Company has been investigated by the 
Research Center for Group Dynamics during a period of over twenty years, as has 
the Detroit Edison Company been involved In a long-term research relationship 
with Floyd Mann and his colleagues at the Institute for Social Research. In 
England, possible sampling biases emanating from continuous Investigation can 
be noted in the Glacier Project where the principal investigator, Elliot Jaques 
of the Tavistock Institute, has maintained research contact for almost twenty 
years. 

A second source of unknown sampling bias is introduced into industrial 
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research in the process of securing permission from selected organizations to 
engage in research. For example, six of the twelve firms we approached refused 
us permission outright to undertake our research. Three of these firms had 
other research in progress, while the other three gave various reasons for their 
lack of support. For example, one engineering firm which was experiencing 
difficulties in management- shop floor communication was willing to let us 
undertake our research only on the condition that we conduct in-depth interviews 
of their shop floor workers. The net effect of these six refusals was that 
further biases were introduced into our research as a result of the strong 
values and predispositions of organizational ^gatekeepers.* Thus, our findings 
are not generallzable even to firms known to be supportive of social research. 
Again though, our experience in this regard is not unique, but Illustrative 
of other industrial research in which problems of access have been encountered 
(Delany, 1960). 

A third source of sampling bias Is Introduced in the process of getting 
industrial workers within selected firms to participate In one's research. 
Here also, our experiences are illustrative. Sixty-one percent of the 5,274 
potential respondents participated in our survey by returning a completed 
questionnaire, with company response rates varying from 37% to 83%. A whole 
host of uncontrollable factors influenced these response rates* For example, 
during the time between one company's acceptance of our survey and the actual 
distribution of questionnaires, a works dispute reaching the national level ^ 
occurred. At the time we arrived at the factory with our questionnaires, neither 
the personnel director nor the chief union officials were in the plant. They 
were In conference in another city. Unfortunately, these were the only 
people in addition to senior management who were knowledgeable of our research 
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and when it was to take place. Although we had agreed that they would 
advertise our research » understandably they had not, and a good part of 
our first day was spent in explaining who we were and why we were in the plant. 
On the second day, during the worst snow storm of the winter, management 
and union officials returned from their conference, whereupon the unions 
instituted an overtime ban, and we began distributing our questionnaires. 
On the following day, the unions wefre threatening to walk out and we were 
walking around the factory familiarizing ourselves with production technologies. 
Meanwhile, the snow was beginning to take its toll. As it accumulated, 
absenteeism of up to 25% occurred in some departments, and we began collecting 
our questionnaires. Needless to say, the urgency of our ''scientific research*' 
was not felt on the shop floor, or by management for that matter. The fact 
that we managed to obtain even a 37% response in this firm is, in retrospect, 
somewhat surprising. 

Without belaboring the point, it should be clear that a multitude of 
uncontvollable, mundane factors such as the industrial relations environment 
and even the weather, can serve to influence to a great extent whether or not 
industrial researchers are blessed with high returns. More generally, industrial 
research is not undertaken in a vacuum, but in complex socio-cultural settings. 
One major consequence is that in deciding which firms to research, in gaining 
entree to these firms, and in getting industrial workers to participate, 
researchers tjrpicaily encounter numerous unforeseen events which introduce 
unknown sampling biases into their research. However, we noted above that to 
the extent that a research includes such biases. Its evaluation becomes 

problematic* The intrinsic nature of Industrial research thus poses a frus- • 
tratlng problem for researchers concerned to generate reliable conclusions 
about industrial life. 
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The obvious question is, how do industrial researchers deal with this 
problem? Perhaps the most common ''solution" adopted has been to use signif- 
icance tests in the evaluation of industrial research findings* However, In 
that this solution assumes random samplins in its interpretation, it is probably 
the least adequate of all currently used strategies (see» Galtung, 1967:340- 
89; Morrison and Henkel, 1970)/ Consequently, in most cases, the researcher 
is faced with the fundamental problem of getting a grip on unknown and uncon- 
trollable sampling bias in order to evaluate his findings* 
Toward Reliable Knowledge About Industrial Life 

Having reached this point. It should be clear that Industrial researchers 
face a hoary problem in evaluating their research. However, this does not 
mean that their research is unimportant or that reliable conclusions about 
industiiai life cannot be reached. Quite to the contrary, it simply means 
that the systemic qualities of research In complex socio-cultural systems 
preclude reaching reliable conclusions on the basis of the findings of any 
single study. 

It has long been recognized by many social scientists that reliable 

conclusions about social life will emerge only as the findings of numerous 

and Independent studies are accumulated. To quote Flnifter (1972:120-21): 

''Ultimately, *..the most powerful and trustworthy response 
researchers have been able to muster to guard against the 
vagaries of chance and artifact is to cumulate evidence... 

If a result can be made to reappear on repeated , independent 
occasions, our basis for placing confidence in it broadens 
accordingly." 

The reason for this is that, by pooling and analyzing the resulting 
distribution of findings from a large number of independent studies » the 
researcher can determine what effects, if any, unknown sampling biases have 
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had on researches in an area. If the distribution of pooled findings converges 
toward a limit, then this limit can be taken as the "true*' result of researches 
in the area, and it reasonably can be concluded that sampling biases are random 
in their effects. Of course, if this distribution does not approach a limit, 
i.e., if it turne out to be flat, bi-modal, or multi-modal, then findings can 
be re-analyzed introducing controls for potentially important variations 
between studies. 

One major constraint on this procedure is that it is necessary that a 
large number of separate studies be available in a given area. Fortunately for 
industrial researchers, this is the case. In libraries, research centers, and 
data banks around the world, large groups of separate researches are currently 
available on problems of importaace to industrial social scientists, e.g., 
productivity, absenteeism, retention, job satisfaction, etc. 

A second constraint is that industrial researchers concerned to pool 
findings on such topics will find little in the literature or in the past 
experiences of their colleagues to help them solve their specific problems* 
Although pooling is obviously an important task, we are only beginning to come 
to grips with the specific theoretical, pragmatic, and technical problems 
involved in cumulating research findings. In a companion paper (Taveggia, 1974), 
we address these problems and discuss how a cumulation was accomplished and to 
what effect for 74 separate studies concerned with the relative effectiveness 
of different methods of college teaching and 24 individual studies involving 
the relative effects of maternal employment on child development. 

Perhaps the most fundamental constraint on pooling research findings is 
that this procedure does not solve the problems of researchers concerned to 
evaluate immediately their research findings* Instead, it requires researchers 
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to postpone evaluation until such time as their findings are pooled with the 
findings of numerous other researches. However, to the extent that our 
analysis of industrial research has been accurate one, it is clear that we 
may have no Justifiable alternative* 

We may conclude by stating briefly some of the implications of data 
cumulation should it become a widely accepted strategy in industrial research* 
First, the consequences for theory building are tremendous. Theoretical models 
based on substantial and exhaustive independent empirical bases will likely 
result in powerful explanatory tools, certainly more adequate than many of 
the competing post hoc theories currently in existence in industrial psychology 
and sociology, most of them based on the results of one or , at best, a few 
empirical studies. Second, the implications for research are also important 
in that the cumulative aspect of science will be given more emphasis. Where* 
as pioneering studies are essential to the growth of a discipline, cumilajtion 
is necessary for its consolidation and elaboration. Finally, «.he importance 
of data pooling for industrial research applications cannot be underestimated. 
This strategy will allow estimates of confidence to be made in the application 
of research findings with the result that much of the faddlshness associated 
with industry could be eliminated. In short, the Introduction of data pooling 
as a stage in the research process is absolutely essential if one of our goals 
as industrial researchers is to generate reliable and valid conclusions about 
industrial life. 



ERLC 



10. 



References 



Uelany, W. Some field notes on the problem of access In organizational research. 
Administrative Science Quarterly , 1960, 5^ 448-57. 

Finlfter, B. M. The generation of confidence: Evaluating research findings 
by random subsample replication. In H. L. Costner (Ed.)> Sociological 
Methodology . San Francisco: Josey-Bass, 1972, p. 112-175. 



Galtung, J. Theory and methods of social research . New York: Columbia 

University Press, 1967. 
Morrison, D. E. and Henkel, R. E. (Eds.) The slRnificance test controversy: 

A reader . Chicago: Aldine, 1970. 
Scott, W. R. Field methods in the study of organizations. In A. Etzlonl 

(Ed.), A Sociological Reader on Complex Organizations . New York: 

liolt, Rlnehart, and Winston, 1969. 
Taveggla, T. C. Resolving research controversy through empirical cumulation. 

Sociological Methods and Research , _2, in press. 




\ 



ERLC 



